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1. INTRODUCTION 

Currently, the Industrial Revolution 4.0 is driving learning by leveraging technology [1] that was 
originally based on face-to-face in the classroom to become an online system. This is expected to help 
students obtain and process information quickly [2]. The lack of learning systems using technology is also a 
problem [3]. Conventional approaches and methods used so far should be combined with the use of 
interactive technology [4] because the use of traditional learning approaches makes students passive [5]. 

The use of technology is also necessary to support the learning process, such as planning, 
implementation, and evaluation. Among the three, evaluation is also a very important part as it provides an 
overview of the achievement of learning objectives for the material presented [6]. Evaluation using 
technology will make learning more interesting, effective and efficient so that it can increase students' 
learning motivation [7], [8]. The accuracy of the student's proficiency data is highly dependent on the 
accuracy of the assessment tool and the learning process [9]. With technology, teachers can instantly show 
students’ learning outcomes when the learning process is complete [10]. 
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The results of interviews with chemistry teachers at SMA (Senior High School) Muhammadiyah 1 
Pontianak, Pontianak, Indonesia provide information that the evaluation of the learning process is done 
manually in the form of multiple-choice questions and/or essays with pen and paper and then collected within 
a certain time limit. One of the materials in the odd semester in class XI MIA is acid-base material with the 
lowest percentage of incompleteness, which is 56.12%. This material must be understood by students to 
understand advanced materials such as salt hydrolysis and buffer solutions. Therefore, teachers often take a 
long time to explain acid-base material and rarely do end-of-class evaluations due to time constraints. The 
result that occurs is a lack of information about the mastery of the presented substance. 

The limited time to evaluate the learning process can be overcome by making use of online-based 
technological developments. Traditional evaluation methods cannot generate student motivation and 
evaluation is less effective [11]. Online evaluation is expected to provide accurate and rapid evaluation 
results, making it easier for teachers and saving time in evaluating learning [12], [13]. One of the assessment 
tools that can be developed in evaluating learning is Kahoot! application. 

Kahoot! is an online quiz application in the form of test questions that can be developed and 
presented in a game format [14]-[16]. Points are awarded to students who answer correctly and students 
involved in the game are listed in the player list. The advantage of this Kahoot! includes an element of 
competition, the quiz results are displayed directly on the class screen so that it can be used as a learning 
motivation for students [17] when earning points and can be used through various media including 
computers, tablets, and androids [18]. This kind of evaluation process makes the learning process be fun [19], 
[20]. 

Kahoot! has four functions namely games, quizzes, discussions, and surveys. For games, there is an 
option to create the question type and determine the most appropriate answer and the time it takes to answer 
the question. The answers are displayed with pictures and colors. The students are asked to choose a 
color/image that represents the answer [21]. 

Several studies that have been conducted can conclude that the use of media, games, and assessment 
tools in the developed Kahoot!-based learning process meets the good criteria and is suitable to use as a tool 
to assess learning in schools [22]—[24]. Based on these issues, this research is expected to bring benefits to 
students, teachers, and schools regarding the Kahoot!-based learning assessment tool and provide a facility to 
become creative and innovative chemistry teachers that also can motivate students to advance their chemistry 
knowledge. Therefore, this research aims to produce an evaluation instrument based on Kahoot! suitable for 
use in chemical studies on acid-base materials. 


2. RESEARCH METHOD 

This research was conducted at SMA Muhammadiyah 1 Pontianak, Pontianak, Indonesia on 
students of eleventh grade students of mathematics and science using ( XI MIA) a quantitative approach. The 
research procedure refers to the Borg and Gall model which consists of 10 steps [25]. However, this research 
has only been carried out up to the seventh stage because it has describes the instrument is valid, practical, 
and reliable [26]. Figure 1 displays the Borg and Gall research procedures. 

The research subjects used in this study were students of class XI MIA 1, XI MIA 2, and XI MIA 3 
SMA Muhammadiyah 1 Pontianak in academic year of 2019 which amounted to 120 students and had 
received acid-base material. Sampling used a random sampling technique, namely a random sampling 
technique. Field trial sample | involved nine people, namely three students with low, medium, and high 
abilities from class XI MIA 2. While trial 2 involved 29 students from class XI MIA 1 SMA Muhammadiyah 
1 Pontianak. 

The indirect communication and measurement techniques are the data collection techniques in this 
study. The validity and practicality of the Kahoot!-based evaluation instrument were determined with indirect 
communication techniques. The instruments used are questionnaires and validation sheets. The questionnaire 
used is a teacher response questionnaire and a student response questionnaire to the evaluation instrument 
using Kahoot!. The questionnaire used is a Likert scale with 4 rating scales [27] namely: strongly agree (SA), 
agree (A), disagree (D), and strongly disagree (SD). While the validation sheet is used to determine the 
validity of the Kahoot!-based evaluation instrument. The technique of collecting data from the results of 
instrument validation is done by giving experts consisting of two materials experts and ttwo media experts a 
set of Kahoot!-based evaluation tools including a grid of questions, Kahoot!-based questions, and answer key 
guidelines and validation sheets. 

The criteria for the validity and practicality of the instrument are in the percentage 62.5 [28]. The 
results of the reliability calculation are adjusted to the value of the reliability coefficient (r11). Reliability is 
obtained if the value is more than 0.60 [29]. 
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Research and Data Collection 


Figure 1. Borg and Gall research procedures [25] 


3. RESULTS AND DISCUSSION 
3.1. Kahoot!-based evaluation instrument 

Kahoot! is an existing application and is used by more than 30 million users [30]. Researchers only 
develop instruments to evaluate acid-base learning material. This evaluation instrument can be accessed via 
the web or mobile phone as shown in Figure 2 and Figure 3. The Kahoot! parts can be seen in Figure 4. 
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Figure 2. Display of evaluation instruments based Kahoot! on the web Figure 3. Display Kahoot! on 
mobile phone 
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Figure 4. Main components of Kahoot! 


Information: 

1. Teacher account 

2. Kahoot! main page 

3. A collection of questions created by the teacher 

4. Results of the demand evaluation 

5. Enter an evaluation question 

The stages of developing an evaluation instrument based on Kahoot! in this study are as: 


3.1.1. Research and data collection 

At the research and data collection phases, needs analysis, literature review and problem 
identification were performed. The needs analysis aims to determine the basic problem of research problem 
[31]. It conducted on eleventh grade teachers and students of mathematics and science at Muhammadiyah 1 
Pontianak senior high school through observation and interviews on October 29, 2019. The observation 
results showed that students complained during the evaluation at the end of the lesson because they felt tired 
after taking the lesson. Most students chose to play mobile phone instead of following the evaluation 
activities provided by the teacher. The literature review analysis and some relevant previous research results 
regarding the importance of evaluation instruments teachers and students have been carried out. Several 
studies have examined the use of Kahoot! that can help students evaluate learning [2], [14], [26], [32]. 

The problem identification is performed to analyze the factors that cause problems in the evaluation 
process so that new products need to be developed. At this phase, the researchers conducted a field survey to 
gather information about the chemistry learning process at Muhammadiyah 1 Pontianak senior high school. 


3.1.2. Planning 

Through this phase, the researchers made a product design in the Kahoot! on acid-base material. The 
creation a Kahoot!-based evaluation instrument includes a grid of questions, evaluation questions, answer 
key guidelines, and validation sheets. A total of fifteen item of questions are made up based on different 
levels of thinking in Bloom’s Taxonomy. There are five questions of remembering level (C1), seven 
questions of understanding level (C2), and three questions of applying level (C3). 


3.1.3. Product draft development 

At this phase, material validation and media validation were performed on the Kahoot!-based 
evaluation instrument. The results of the material expert validation recapitulation are presented in Table 1. 
The results of material expert validation in Table 1 show that first validator and second validator achieve a 
percentage of 81.73% (very valid). The two material expert validators offered suggestions to improve some 
of these shortcomings, in particulars: i) Fixed the thinking aspect for question number 7 changed from C2 to 
C3; ii) Fixed writing compound index on question number 8; and iii) Fixed writing a dot at the end of the 
question. The complete suggestions show in Table 2. The results of the validation on media experts showed 
that the Kahoot!-based evaluation tool could be used for field trials with the revision requirements as per the 
suggestions/inputs of the two validators. The results of the media expert summary can be seen in Table 3. 
Table 3 shows that the final percentage of material validation results of the two validators is very valid. As 
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for suggestions and improvements from media experts, namely adding time to work on questions. To work 
on the problem of counting, so the processing time is recommended to be changed from 120 to 240 minutes. 


Table 1. Recapitulation of material expert validation 


i ee Score Percentage (%) 
Aspeci ios puke Validator I Validator II j 
Presentation Concept coherence 3 3 75 
Completeness of question identity 4 3 87.5 
The question presented is appropriate with the 
presentation of questions according to basic 4 3 87.5 
competence and indicators 
Clarity of image presentation 4 4 100 
Can be used individually or in groups 3 3 75 
As a practical and efficient evaluation instrument 3 3 75 
The suitability of the answer key with questions 4 3 87.5 
Quality of the question Completeness of questions according to material 3 3 75 
Communicative 3 3 75 
Encourage curiosity 3 3 75 
Construction The suitability of the question based on the 
TR 3 3 75 
students' ability 
Order of presentation of the question 4 4 100 
Provide learning motivation 3 3 75 
Average 81.73 
Criteria Very valid 
Table 2. Materials experts suggestions and improvements 
No. Suggestions and improvement results 


1 The thinking aspect in question 7 must be improved because in question 7 the thinking level of students has been 
entered in the application, namely C3. Question number 7 is not only limited to counting, but students must also be 
able to examine the question because, in question number 7, 2 connections must be identified before students 
answer the question. 

2 Problem numbers need to be corrected as the connection is not written correctly. The compound N2H4is composed 
of 2 nitrogen atoms and 4 hydrogen atoms, so the writing for the nitrogen and hydrogen indices should be smaller 
and lower than or commonly referred to as the subscript. 

3 The writing a dot at the end of the question should be corrected because there are correct Indonesian language rules 
when writing the question. As for the written language in written questions, it is guided by the rules of Enhanced 
Spelling (EYD), namely, there are several things to consider, such as capitalization and the use of periods (.). The 
dot at the end of the question-shaped question is 4, while the dot in the middle of the inserted question is 3 pieces. 


Table 3. Recapitulation of media expert validation percentage 


: Aaa ey Score Percentage (%) 
Aspect DPescuption Validator I Validator II 

Presentation Instructions for use are clear 3 4 87.5 
The display of the values obtained is clear 4 3 87.5 
Interesting presentation of the questions 3 3 75 
Can be used individually or group 4 3 87.5 
Interesting answer key display 3 3 75 

Content design Appropriate color display 3 4 87.5 
Various question 3 3 75 
The quality of photos or images is good 3 3 75 
Characters or letters are appropriate 3 3 15 

Design The layout of elements is appropriate 3 4 87.5 
The website uses the appropriate characters or letters 3 4 87.5 
Attractive application display 3 3 75 
Provide learning motivation 3 3 75 

Ease of use Easy to access the application 3 3 75 
Practical use of evaluation tools 3 3 15 
The menu and facilities (buttons) are easy to 3 4 87.5 
understand 

Average 80.47 

Criteria Very valid 


3.1.4. Initial field trial 
The first field trials were also conducted by providing a questionnaire for the responses of teachers 
and students who became the research sample after conducting the trial. This questionnaire was provided to 
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test the practicality of the Kahoot!-based evaluation instrument. The recapitulation of the results of the 
teacher's response questionnaire and the responses of the students in the initial field trials can be seen in 
Table 4. In the initial field trial, the reliability of the questions was determined from the results of the 
evaluation of the questions using Kahoot!. Based on the calculations, the value of reliability (r11) of the 
questions obtained is 0.89. The number above 0.60 indicates the instrument can be used in the main field 
trial. 


Table 4. Recapitulation of the percentage of teacher and student responses to the questionnaire for the initial 
field trials 


Respondent Total score Percentage score Criteria 
Teachers 30 85.80% Very practical 
Student 278 83.33% Very practical 
Average 84.56% Very practical 


3.1.5. Revision of the initial field trials 

The revisions made at this stage were based on the analysis of comments/suggestions provided by 
teachers and students about the Kahoot! based evaluation instrument. The teachers said the evaluation 
instruments are good for use in the learning process. However, writing connections in the problem should be 
improved. The students also said that the time for solving the questions should be synchronized with the 
questions. Even so, they argued that the Kahoot! accelerate or make it easier for them to access learning. It is 
also very interesting to use as a learning evaluation 


3.1.6. Main field trial 

The main field trial was conducted to determine the practicality of Kahoot! by providing a 
questionnaire on the responses of students and teachers. The results of the questionnaire responses of teachers 
and students in the main field trials can be seen in Table 5. In the main field trial, the reliability of the 
questions was also determined based on the results of the evaluation of the questions with Kahoot!. Based on 
the results of the calculation of the reliability of the questions obtained, the value of r11 is 0.82. It means, the 
evaluation of the questions based on the Kahoot! is very reliable. 


Table 5. Recapitulation of the results of the responses of teachers and students in the initial field test 


Respondent Total score Percentage score Criteria 
Teacher 35 97.29% Very practical 
Student 900 86.21% Very practical 

Average 81.71% Very practical 


3.1.7. Product improvement of field trial results 

At this stage, students are asked to provide comments/suggestions about the Kahoot!-based 
evaluation instrument. Recapitulation of students' responses in the main field trials can be seen in Table 6. 
Overall students’ responses to the main field trials showed positive comments about the Kahoot! -based 
evaluation instrument. Therefore, the Kahoot! -based assessment tool is no longer under review and is 
considered as a final product that can be used in the learning process. 


Table 6. Recapitulation of the results of comments/suggestions of the teachers and the students in main field 
trials 
Respondent Conclusion 

Teacher I find the designed evaluation instrument very good and very interesting for students 
to read. So this evaluation instrument can be used to support the learning process 
activities. 

Students This Kahoot! application is good for evaluation in learning. 
Increase it, hopefully, the Kahoot! application will be more developed. 
The appearance of the Kahoot! -based evaluation instrument is very interesting and 
makes students motivated to evaluate the learning. 
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3.2. Feasibility analysis 

The feasibility analysis of the Kahoot!-based evaluation instrument was performed to test the 
validity, practicality, and reliability of the developed product. These three aspects serve as benchmarks to 
determine whether a Kahoot!-based evaluation instrument is suitable for use or vice versa. 


3.2.1. Validity analysis 
3.2.1.1. Material expert validation 

The material expert stated that the Kahoot!-based evaluation instrument was usable after revisions 
were made based on suggestions and improvements from the two validators. The recapitulation results of 
material expert validation on each aspect can be seen in Figure 5. 


Figure 5. Percentage of results of material expert validation on each aspect 


Figure 5 shows that the results of material expert validation are feasible to use, but several aspects 
still need to be developed, such as aspects of question quality, completeness of questions according to the 
material, communication skills, and arousing student curiosity. The two validators also give a percentage of 
75%, which means that it is still not perfect because the validator's rating has not reached a percentage of 
100%. Overall, the results of the percentage of materials experts showed a value of 81.71%. This means 
Kahoot! met the validity criteria with very valid criteria [26]. 


3.2.1.2. Validation of media experts 

The media experts state that the Kahoot! -based evaluation instrument was usable after revisions 
according to the suggestions and improvements of the two validators. The percentage results of validation of 
media experts in each aspect can be seen in Figure 6. 


82.50 
81.25 
79.17 
78.12 E 
Presentation Content design Design Easy to use 


Figure 6. The percentage of media validation results from content experts in each aspect 


Figure 6 shows the percentage of validation results from content experts in four aspects contain 
presentation, content design, design dan easiness of media has shortcomings. There are not considered 
perfect in terms of media because not every aspect such as aspects of presentation, content design, and ease 
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of use has reached 100%. Therefore, it is necessary to improve the presentation of questions, color rendering, 
and image quality on Kahoot! based evaluation instruments. Although there are deficiencies in the 
assessment of the media validation of the Kahoot!-based assessment instrument, based on the final results, 
the percentage of media experts demonstrate that the Kahoot!-based assessment tool is in line with the 
validity criteria [12], with the final result the percentage of media expert validation is 80.47% (very valid). 


3.2.2. Practical analysis 

The practical aspects can be derived from the analysis of teacher-responder questionnaires and 
student questionnaires to the use of Kahoot!-based assessment instruments. The recapitulation of the results 
can be seen in Table 7. The table response shows the of practicality Kahoot! based assessment tools are 
categorized as practical if the average of teacher and student responses value is equal to or more than 61%. 


Table 7. Results of questionnaire response initial field trials and main field trials 
Respondent value 


Teacher Student 
Initial field trials 83.33% 85.80% 
Main field trials 97.22% 86.21% 
Average 90.27% 86.02% 


4. CONCLUSION 

The Kahoot!-based evaluation instrument developed in this study is worthy of use as evaluation 
material in the acid-base learning process. It meets the criteria of validity, practicality, and reliability. 
However, further research needs to be done in the diffusion stage and developed not only on acid-base 
materials but also on other materials. 
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